Report on the Second Second Challenge on Generating Instructions in Virtual Environments (GIVE-2.5)
نویسندگان
چکیده
GIVE-2.5 evaluates eight natural language generation (NLG) systems that guide human users through solving a task in a virtual environment. The data is collected via the Internet, and to date, 536 interactions of subjects with one of the NLG systems have been recorded. The systems are compared using both task performance measures and subjective ratings by human users.
منابع مشابه
Report on the Second NLG Challenge on Generating Instructions in Virtual Environments (GIVE-2)
We describe the second installment of the Challenge on Generating Instructions in Virtual Environments (GIVE-2), a shared task for the NLG community which took place in 2009-10. We evaluated seven NLG systems by connecting them to 1825 users over the Internet, and report the results of this evaluation in terms of objective and subjective measures.
متن کاملReport on the First NLG Challenge on Generating Instructions in Virtual Environments (GIVE)
We describe the first installment of the Challenge on Generating Instructions in Virtual Environments (GIVE), a new shared task for the NLG community. We motivate the design of the challenge, describe how we carried it out, and discuss the results of the system evaluation.
متن کاملThe Heidelberg GIVE-2 System
This paper describes a NLG system, developed by students at Heidelberg University, that generates instructions that direct users through virtual environments. The NLG architecture is implemented as a modular pipeline consisting of elementary components. The system has been developed for participation in the 2nd installment of the Generation Instructions in Virtual Environment (GIVE) challenge.
متن کاملThe Software Architecture for the First Challenge on Generating Instructions in Virtual Environments
The GIVE Challenge is a new Internetbased evaluation effort for natural language generation systems. In this paper, we motivate and describe the software infrastructure that we developed to support this challenge.
متن کاملThe Thumbs Up! Twente system for GIVE 2.5
This paper describes the Thumbs Up! Twente system, a natural language generation system designed for the GIVE 2.5 Challenge. The purpose of the system is to guide a user through a virtual 3D environment by generating instructions in real-time. Our system focuses on motivating the user to keep him playing the game and trying to find the trophy.
متن کامل